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BACKGROUND OF THE INVENTION 
The present invention relates to a motion video encoding device 
and a motion video encoding method, and in particular, to a motion video 
encoding device and a motion video encoding method capable of encoding 
5 video signals so as to reduce the degradation of video data caused by 
o errors when coded video signals (compressed video signals, for example) 

ZES. 

s : 

~£ are transmitted via transmission lines of low circuit quality. 
q Description of the Related Art 

l] Various encoding methods have been employed for motion video 

1_ 10 encoding devices. In order to encode and transmit each frame (each 
N input image) of input video data, a conventional motion video encoding 
%j device adaptively makes a selection from intra-frame encoding (INTRA) 
S]" and inter-frame prediction encoding (INTER) for each block (as a unit of 
encoding) and uses the selected encoding mode for encoding the block. 
15 The conventional motion video encoding device makes the encoding mode 
selection for each block based on coding efficiency. In addition to the 
encoding mode selection, the conventional motion video encoding device 
carries out refresh (forced refresh), that is, forcedly encodes a block by 
means of the intra-frame encoding, in order to reduce the degradation of 
20 image quality caused by errors occurring during the transmission of the 
coded video data. 

In the inter-frame prediction encoding, if the image quality 
degradation (image degradation) occurred to the coded data due to errors 
in signal transmission, similar degradation occurs to the next frame 
25 (which is decoded by reference to the degraded frame). As the 
probability of the error-caused degradation in a referred area of the 
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* transmitted frame increases, the probability of serious degradation in 
the next frame increases. By such propagation of degradation through 
frames, subjective image quality is degraded considerably. On the other 
hand, by forcedly carrying out the refresh to the image of a frame at 
5 some midpoint, the propagation of degradation can be avoided and 
thereby resistance to image degradation can be improved. The degree of 
degradation when an error occurred to a specific area of an image 
depends on the distribution of pixel values in the image. Therefore, by 
u z carrying out the forced refresh with high priority to blocks from which 
S 10 serious image degradation is expected, the resistance to errors can be 
improved effectively without deteriorating coding efficiency much. 

Fig.l is a block diagram showing a conventional motion video 
encoding device which has been disclosed in Japanese Patent Application 
Laid'Open No. 2000-165881. The conventional motion video encoding 
15 device of Fig.l includes a frame memory 10, a degradation power 
calculation section 12, a block encoding section 18, a mode selection 
section 24, a code volume control section 25, a counter 26, a data loss 
probability estimation section 27, a refresh priority calculation section 28, 
and a refresh block selection section 29. 
20 In the motion video encoding device of Fig.l, an input image 

(current frame) 101 and a reference image (reference frame) 102 are 
stored in the frame memory 10. The reference image 102 is supplied 
from the block encoding section 18 as a decoded image of a previous 
frame. The block encoding section 18, which encodes each block of the 
25 frame as will be explained later, is also provided with a decoding function. 
By the decoding function, the block encoding section 18 decodes coded 
data of a previous frame and sends the decoded image to the frame 
memory 10 as the reference image 102. 

The mode selection section 24 selects the encoding mode for each 
30 block of the input image 101 from the intra-frame encoding (INTRA) and 
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the inter-frame prediction encoding (INTER) in consideration of the 
coding efficiency based on the complexity etc. of the block. The code 
volume control section 25 sets a quantization step (a step width 
employed for quantizing image data) for each block so that coding 
5 distortion (artifact) will be minimum when the input image 101 is 
encoded by the encoding mode selected by the mode selection section 24 
under the condition that the volume of generated code should be a preset 
volume or less. 

The degradation power calculation section 12 assumes a case 

10 where transmission error occurred to coded data of a block, and 
calculates an error power (degradation power) between a complemented 
image (an image complemented with lost data) and the original image. 
The counter 26 monitors the coded data to be transmitted and thereby 
counts the volume of code from the latest synchronization code pattern 

15 (synchronization code) to a block as the target of encoding (hereafter, 
referred to as "target block"). The data loss probability estimation 
section 27 receives the count of the counter 26 (code volume from the 
latest synchronization code pattern to the target block), and estimates 
the probability of data loss occurring to each block due to transmission 

20 error, based on the count. 

In this case, the input to the counter 26 is delayed for a frame, 
therefore, the counter 26 outputs the count with regard to coded data of a 
frame before. In other words, the counter 26 counts the code volume 
from the latest synchronization code pattern in the previous frame to the 

25 target block in the previous frame (i.e. a block in the previous frame at 
the same position as the target block) and gives the count to the data loss 
probability estimation section 27. The data loss probability estimation 
section 27 estimates the data loss probability for the target block of the 
current frame, based on the code volume to the target block of the 

30 previous frame. 



Figs. 2 A through 2C are schematic diagrams for explaining the 
relationship between the data loss probability and the point when 
transmission error occurred. As shown in Fig.2A, synchronization code 
patterns (synchronization codes) are inserted in the intervals between 
coded data (each of which including a plurality of blocks). As shown in 
Fig.2B and Fig.2C, a decoder (unshown) receiving the coded data 
discards data from the error occurrence point to the next synchronization 
code pattern, and restarts data decoding from the next synchronization 
code pattern. Therefore, if transmission error occurs at random, the 
probability of the data discard increases as the coded data gets far from 
the latest synchronization code pattern. Assuming such decoding 
operation of the decoder in cases of errors, the data loss probability 
estimation section 27 estimates the data loss probability higher as the 
distance from the latest synchronization code pattern gets longer. 

The refresh priority calculation section 28 calculates and sets the 
priority concerning the forced refresh by making threshold judgments 
with regard to the degradation power of the current frame (which is 
obtained by the degradation power calculation section 12) and the data 
loss probability of each block of the current frame (which is obtained by 
the data loss probability estimation section 27), and sends the refresh 
priority to the refresh block selection section 29. 

The refresh block selection section 29 selects blocks to which the 
forced refresh (intra-frame encoding) should be executed, based on the 
refresh priority supplied from the refresh priority calculation section 28. 
The block encoding section 18 encodes each block of the current frame 
according to the encoding mode determined by the mode selection section 
24 and the refresh block selection section 29 and the quantization step 
determined by the code volume control section 25, and thereby generates 
the coded data. 

In the conventional motion video encoding device, the forced 
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refresh is carried out with higher priority to blocks having high 
degradation power and high data loss probability, since the degree of 
image degradation gets higher if error occurs to such blocks. 

However, in the conventional motion video encoding device, the 
5 selection of the forcedly refreshed blocks is made independently of the 
switching of the encoding mode (between the intra-frame encoding and 
the inter-frame prediction encoding considering the coding efficiency), 
therefore, depending on the setting of the threshold values used for the 
mode selection, the intra-frame encoding might be selected for too many 
jh 10 blocks, causing deterioration of coding efficiency 

'% Further, in the conventional motion video encoding device, actual 

jt; image degradation in cases where transmission error occurred is not 
N: reflected sufficiently, and the selection of the forcedly refreshed blocks is 
■ not made effectively. Especially, the effects of the propagation of 

H 15 degradation due to the inter-frame prediction encoding are not taken 

£ . 

k\ into consideration. 
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SUMMARY OF THE INVENTION 
It is therefore the primary object of the present invention to 
20 provide a motion video encoding device and a motion video encoding 
method, by which the forced refresh can be carried out properly and 
effectively, and thereby image degradation can be reduced without 
deteriorating the coding efficiency. 

In accordance with a first aspect of the present invention, there is 
25 provided a motion video encoding device which adaptively selects and 
uses intra-frame encoding or inter-frame prediction encoding for 
encoding each block as an encoding unit, comprising frame memory 
means, block encoding means, code counting means, code volume control 
means, data loss probability estimation means, frame coding distortion 
30 estimation means, degradation power calculation means, degradation 
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estimation calculation means and mode selection means. The frame 
memory means stores an input image and a reference image as the result 
of decoding of a frame before. The block encoding means refers to the 
input image stored in the frame memory means and carries out a block 
5 encoding process for each block of the input image according to a selected 
encoding mode and a quantization step designating the width of 
quantization of image data, and thereby generates coded data. The code 
counting means monitors the coded data and thereby counts the code 
volume from the latest synchronization code pattern inserted in the 
JflO coded data to coded data of a block that has been encoded latest. The 

O code volume control means refers to the code volume of the coded data 

4- 

til outputted by the block encoding means and a pixel value distribution 

fn 

LI statistic obtained by analyzing the input image and the reference image 
stored in the frame memory means, thereby calculates the quantization 

g 15 step for each block and for each selectable encoding mode so that image 

^ distortion caused by the block encoding process will be minimum, and 

D outputs a code volume predicted value assigned to a target block which 

\*~ 

will be encoded next, under the condition that the code volume for a 
frame should be a preset volume or less. The data loss probability 

20 estimation means estimates the probability that data loss will occur to 
the target block due to transmission error, based on the code volume 
predicted value assigned to the target block, the code volume counted by 
the code counting means, and a preset error probability per bit. The 
frame coding distortion estimation means estimates frame coding 

25 distortion as image distortion caused by the block encoding processes for 
the frame, by referring to the pixel value distribution statistic of the 
input image obtained by the code volume control means. The 
degradation power calculation means calculates degradation power as 
error power between the input image and a decoded image assuming that 

30 data loss occurred to the target block during data transmission of the 



coded data, by use of the input image or the reference image stored in 
the frame memory means. The degradation estimation calculation 
means calculates a degradation estimation as an expected value of image 
degradation occurring to the target block due to data loss caused by 
transmission error, for each block and for each selectable encoding mode, 
based on the degradation power calculated by the degradation power 
calculation means and the data loss probability estimated by the data 
loss probability estimation means. The mode selection means selects an 
optimum encoding mode for the target block by referring to the 
degradation estimation calculated by the degradation estimation 
calculation means and the frame coding distortion estimated by the 
frame coding distortion estimation means. 

In accordance with a second aspect of the present invention, in the 
first aspect, the degradation power calculation means complements pixel 
values of the target block with pixel values of surrounding pixels of the 
input image, and regards error power between the complemented image 
and the input image as the degradation power. 

In accordance with a third aspect of the present invention, in the 
first aspect, the degradation power calculation means complements pixel 
values of the target block with pixel values of a block of the reference 
image at the same position as the target block, and regards error power 
between the complemented image and the input image as the 
degradation power. 

In accordance with a fourth aspect of the present invention, in the 
first aspect, the degradation power calculation means complements pixel 
values of the target block by copying pixel values from a motion 
compensation area of the reference image corresponding to the target 
block, and regards error power between the complemented image and the 
input image as the degradation power. 

In accordance with a fifth aspect of the present invention, in the 
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first aspect, the data loss probability estimation means estimates the 
probability that data loss will occur to the target block due to 
transmission error, by adding the code volume predicted value assigned 
to the target block by the code volume control means to the code volume 
counted by the code counting means and multiplying the sum by the 
preset error probability per bit. 

In accordance with a sixth aspect of the present invention, in the 
first aspect, the degradation estimation calculation means obtains the 
degradation estimation in the case of the intra-frame encoding by 
multiplying the degradation power calculated by the degradation power 
calculation means by the data loss probability estimated by the data loss 
probability estimation means. The degradation estimation calculation 
means obtains the degradation estimation in the case of the inter-frame 
prediction encoding by obtaining the product of the degradation power 
and the data loss probability, obtaining a degradation propagation term 
as a degradation estimation of a referred area of the reference image 
which is referred to for a motion compensation process in the inter-frame 
prediction encoding, and adding the degradation propagation term to the 
product. 

In accordance with a seventh aspect of the present invention, in 
the first aspect, the degradation estimation calculation means obtains 
the degradation estimation in the case of the intra-frame encoding by 
multiplying the degradation power calculated by the degradation power 
calculation means by the data loss probability estimated by the data loss 
probability estimation means. The degradation estimation calculation 
means obtains the degradation estimation in the case of the inter-frame 
prediction encoding by obtaining the product of the degradation power 
and the data loss probability, obtaining a degradation propagation term 
as a weighted average of degradation estimations of blocks overlapping 
with a referred area of the reference image which is referred to for a 
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motion compensation process in the inter-frame prediction encoding, and 
adding the degradation propagation term to the product. 

In accordance with an eighth aspect of the present invention, in 
the first aspect, the degradation estimation calculation means obtains 
5 the degradation estimation in the case of the intra-frame encoding by 
multiplying the degradation power calculated by the degradation power 
calculation means by the data loss probability estimated by the data loss 
probability estimation means. The degradation estimation calculation 
means obtains the degradation estimation in the case of the inter-frame 
□ 10 prediction encoding by obtaining the product of the degradation power 
P and the data loss probability, obtaining a degradation propagation term 
f] as a degradation estimation of a referred area of the reference image 
N which is referred to for a motion compensation process in the inter-frame 

prediction encoding, multiplying the degradation propagation term by a 
M 15 proportionality constant, and adding the multiplied degradation 
Ci propagation term to the product. 

D In accordance with a ninth aspect of the present invention, in the 

K 

first aspect, the mode selection means obtains the degradation 
estimation of the target block in each candidate encoding mode from the 
20 degradation estimation calculation means, obtains the frame coding 
distortion in each candidate encoding mode from the frame coding 
distortion estimation means, and selects a candidate encoding mode that 
minimizes the sum of the degradation estimation and the frame coding 
distortion as the optimum encoding mode for the target block. 

25 In accordance with a tenth aspect of the present invention, in the 

first aspect, the motion video encoding device further comprises block 
coding distortion estimation means for estimating block coding distortion 
as image distortion caused by the block encoding process for the target 
block by referring to the input image and the reference image stored in 

30 the frame memory means and the quantization step calculated by the 
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code volume control means. The mode selection means selects an 
optimum encoding mode for the target block from the intra-frame 
encoding, the inter-frame prediction encoding and another encoding 
mode "skip" in which pixel values are directly copied from a block of the 
5 reference image at the same position as the target block, by referring to 
the degradation estimation calculated by the degradation estimation 
calculation means, the frame coding distortion estimated by the frame 
coding distortion estimation means and the block coding distortion 
estimated by the block coding distortion estimation means. The 
H10 degradation estimation calculation means calculates the degradation 
□ estimation as an expected value of image degradation occurring to the 
m target block due to data loss caused by transmission error, for each block 
and for each of the selectable encoding modes: the intra-frame encoding! 
' the inter-frame prediction encoding! and the skip, based on the 
p 15 degradation power calculated by the degradation power calculation 
H= means and the data loss probability estimated by the data loss 
O probability estimation means. 

In accordance with an eleventh aspect of the present invention, in 
the tenth aspect, the block coding distortion estimation means estimates 
20 the block coding distortion of the target block in the case of the 
intra-frame encoding or the inter-frame prediction encoding, as 
quantization distortion by use of the quantization step calculated by the 
code volume control means. The block coding distortion estimation 
means estimates the block coding distortion of the target block in the 
25 case of the skip, by obtaining error power between the target block and a 
block of the reference image at the same position as the target block and 
regarding the error power as the block coding distortion. The data loss 
probability estimation means estimates the data loss probability of the 
target block in the case of the intra-frame encoding or the inter-frame 
30 prediction encoding, based on the code volume predicted value assigned 
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to the target block, the code volume counted by the code counting means, 
and a preset error probability per bit. The data loss probability 
estimation means estimates the data loss probability of the target block 
in the case of the skip, based on the code volume of a skip code which 
indicates that the encoding mode is the skip, the code volume counted by 
the code counting means, and a preset error probability per bit. The 
degradation estimation calculation means calculates the degradation 
estimation of the target block in the case of the skip, in the same way as 
the case of the inter-frame prediction encoding. 

In accordance with a twelfth aspect of the present invention, in 
the tenth aspect, the mode selection means obtains the degradation 
estimation of the target block in each of the intra-frame encoding and the 
interframe prediction encoding from the degradation estimation 
calculation means, obtains the frame coding distortion in each of the 
intra-frame encoding and the inter-frame prediction encoding from the 
frame coding distortion estimation means, selects an encoding mode that 
minimizes the sum of the degradation estimation and the frame coding 
distortion from the intra-frame encoding and the inter-frame prediction 
encoding as a candidate optimum encoding mode for the target block, 
compares the selected candidate optimum encoding mode with the skip 
by making a comparison between the two encoding modes with regard to 
the sum of the degradation estimation calculated by the degradation 
estimation calculation means and the block coding distortion estimated 
by the block coding distortion estimation means, and selects one of the 
two encoding modes that minimizes the sum as the optimum encoding 
mode for the target block. 

In accordance with a thirteenth aspect of the present invention, 
there is provided a motion video encoding method which adaptively 
selects and uses intra-frame encoding or inter-frame prediction encoding 
for encoding each block as an encoding unit, comprising an image storage 
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step, a block encoding step, a code counting step, a code volume control 
step, a data loss probability estimation step, a frame coding distortion 
estimation step, a degradation power calculation step, a degradation 
estimation calculation step and a mode selection step. In the image 
5 storage step, an input image and a reference image as the result of 
decoding of a frame before are stored in a frame memory. In the block 
encoding step, the input image stored in the frame memory is referred to 
and a block encoding process is carried out for each block of the input 
image according to a selected encoding mode and a quantization step 
10 designating the width of quantization of image data, and thereby coded 
p data is generated. In the code counting step, the coded data is 
monitored and thereby the code volume from the latest synchronization 
code pattern inserted in the coded data to coded data of a block that has 
been encoded latest is counted. In the code volume control step, the 
15 code volume of the coded data generated in the block encoding step and a 
pixel value distribution statistic obtained by analyzing the input image 
and the reference image stored in the frame memory are referred to, 
thereby the quantization step is calculated for each block and for each 
selectable encoding mode so that image distortion caused by the block 
20 encoding process will be minimum, and a code volume predicted value 
assigned to a target block which will be encoded next is outputted, under 
the condition that the code volume for a frame should be a preset volume 
or less. In the data loss probability estimation step, the probability that 
data loss will occur to the target block due to transmission error is 
25 estimated based on the code volume predicted value assigned to the 
target block, the code volume counted in the code counting step, and a 
preset error probability per bit. In the frame coding distortion 
estimation step, frame coding distortion is estimated as image distortion 
caused by the block encoding processes for the frame, by referring to the 
30 pixel value distribution statistic of the input image obtained in the code 
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volume control step. In the degradation power calculation step, 
degradation power is calculated as error power between the input image 
and a decoded image assuming that data loss occurred to the target block 
during data transmission of the coded data, by use of the input image or 
5 the reference image stored in the frame memory. In the degradation 
estimation calculation step, a degradation estimation is calculated as an 
expected value of image degradation occurring to the target block due to 
data loss caused by transmission error, for each block and for each 
selectable encoding mode, based on the degradation power calculated in 
M=10 the degradation power calculation step and the data loss probability 
D estimated in the data loss probability estimation step. In the mode 
X selection step, an optimum encoding mode for the target block is selected 
JJJ by referring to the degradation estimation calculated in the degradation 
y estimation calculation step and the frame coding distortion estimated in 
D15 the frame coding distortion estimation step. 

H In accordance with a fourteenth aspect of the present invention, in 

q the thirteenth aspect and in the degradation power calculation step, 
pixel values of the target block are complemented with pixel values of 
surrounding pixels of the input image, and error power between the 
20 complemented image and the input image is regarded as the degradation 
power. 

In accordance with a fifteenth aspect of the present invention, in 
the thirteenth aspect and in the degradation power calculation step, 
pixel values of the target block are complemented with pixel values of a 
25 block of the reference image at the same position as the target block, and 
error power between the complemented image and the input image is 
regarded as the degradation power. 

In accordance with a sixteenth aspect of the present invention, in 
the thirteenth aspect and in the degradation power calculation step, 
30 pixel values of the target block are complemented by copying pixel values 
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from a motion compensation area of the reference image corresponding to 
the target block, and error power between the complemented image and 
the input image is regarded as the degradation power. 

In accordance with a seventeenth aspect of the present invention, 
5 in the thirteenth aspect and in the data loss probability estimation step, 
the probability that data loss will occur to the target block due to 
transmission error is estimated by adding the code volume predicted 
value assigned to the target block in the code volume control step to the 
code volume counted in the code counting step and multiplying the sum 
£ 10 by the preset error probability per bit. 

O In accordance with an eighteenth aspect of the present invention, 

jp in the thirteenth aspect and in the degradation estimation calculation 
rf step, the degradation estimation in the case of the intra-frame encoding 
y is obtained by multiplying the degradation power calculated in the 
015 degradation power calculation step by the data loss probability estimated 
H= in the data loss probability estimation step, and the degradation 
D estimation in the case of the inter-frame prediction encoding is obtained 
by obtaining the product of the degradation power and the data loss 
probability, obtaining a degradation propagation term as a degradation 
20 estimation of a referred area of the reference image which is referred to 
for a motion compensation process in the inter-frame prediction encoding, 
and adding the degradation propagation term to the product. 

In accordance with a nineteenth aspect of the present invention, 
in the thirteenth aspect and in the degradation estimation calculation 
25 step, the degradation estimation in the case of the intra-frame encoding 
is obtained by multiplying the degradation power calculated in the 
degradation power calculation step by the data loss probability estimated 
in the data loss probability estimation step, and the degradation 
estimation in the case of the inter-frame prediction encoding is obtained 
30 by obtaining the product of the degradation power and the data loss 
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probability, obtaining a degradation propagation term as a weighted 
average of degradation estimations of blocks overlapping with a referred 
area of the reference image which is referred to for a motion 
compensation process in the inter-frame prediction encoding, and adding 
the degradation propagation term to the product. 

In accordance with a twentieth aspect of the present invention, in 
the thirteenth aspect and in the degradation estimation calculation step, 
the degradation estimation in the case of the intra-frame encoding is 
obtained by multiplying the degradation power calculated in the 
degradation power calculation step by the data loss probability estimated 
in the data loss probability estimation step, and the degradation 
estimation in the case of the inter-frame prediction encoding is obtained 
by obtaining the product of the degradation power and the data loss 
probability, obtaining a degradation propagation term as a degradation 
estimation of a referred area of the reference image which is referred to 
for a motion compensation process in the inter-frame prediction encoding, 
multiplying the degradation propagation term by a proportionality 
constant, and adding the multiplied degradation propagation term to the 
product. 

In accordance with a twenty-first aspect of the present invention, 
in the thirteenth aspect and in the mode selection step, the degradation 
estimation of the target block in each candidate encoding mode is 
obtained from the degradation estimation calculation step, the frame 
coding distortion in each candidate encoding mode is obtained from the 
frame coding distortion estimation step, and a candidate encoding mode 
that minimizes the sum of the degradation estimation and the frame 
coding distortion is selected as the optimum encoding mode for the target 
block. 

In accordance with a twenty-second aspect of the present 
invention, in the thirteenth aspect, the motion video encoding method 
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further comprises a block coding distortion estimation step for 
estimating block coding distortion as image distortion caused by the 
block encoding process for the target block by referring to the input 
image and the reference image stored in the frame memory and the 
5 quantization step calculated in the code volume control step. In the 
mode selection step, an optimum encoding mode for the target block is 
selected from the intra-frame encoding, the inter-frame prediction 
encoding and another encoding mode "skip" in which pixel values are 
directly copied from a block of the reference image at the same position 
!10 as the target block, by referring to the degradation estimation calculated 
3 in the degradation estimation calculation step, the frame coding 
1 distortion estimated in the frame coding distortion estimation step and 
I the block coding distortion estimated in the block coding distortion 
estimation step. In the degradation estimation calculation step, the 
f 15 degradation estimation is calculated as an expected value of image 
degradation occurring to the target block due to data loss caused by 
3 transmission error, for each block and for each of the selectable encoding 
modes- the intra-frame encoding; the inter-frame prediction encoding; 
and the skip, based on the degradation power calculated in the 
20 degradation power calculation step and the data loss probability 
estimated in the data loss probability estimation step. 

In accordance with a twenty-third aspect of the present invention, 
in the twenty-second aspect and in the block coding distortion estimation 
step, the block coding distortion of the target block in the case of the 
25 intra-frame encoding or the inter-frame prediction encoding is estimated 
as quantization distortion by use of the quantization step calculated in 
the code volume control step, and the block coding distortion of the target 
block in the case of the skip is estimated by obtaining error power 
between the target block and a block of the reference image at the same 
30 position as the target block and regarding the error power as the block 
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coding distortion. In the data loss probability estimation step, the data 
loss probability of the target block in the case of the intra-frame encoding 
or the inter-frame prediction encoding is estimated based on the code 
volume predicted value assigned to the target block, the code volume 
counted in the code counting step, and a preset error probability per bit, 
and the data loss probability of the target block in the case of the skip is 
estimated based on the code volume of a skip code which indicates that 
the encoding mode is the skip, the code volume counted in the code 
counting step, and a preset error probability per bit. In the degradation 
estimation calculation step, the degradation estimation of the target 
block in the case of the skip is calculated in the same way as the case of 
the inter-frame prediction encoding. 

In accordance with a twenty-fourth aspect of the present 
invention, in the twenty-second aspect and in the mode selection step, 
the degradation estimation of the target block in each of the intra-frame 
encoding and the inter-frame prediction encoding is obtained from the 
degradation estimation calculation step, the frame coding distortion in 
each of the intra-frame encoding and the inter-frame prediction encoding 
is obtained from the frame coding distortion estimation step, an encoding 
mode that minimizes the sum of the degradation estimation and the 
frame coding distortion is selected from the intra-frame encoding and the 
inter-frame prediction encoding as a candidate optimum encoding mode 
for the target block, the selected candidate optimum encoding mode is 
compared with the skip by making a comparison between the two 
encoding modes with regard to the sum of the degradation estimation 
calculated in the degradation estimation calculation step and the block 
coding distortion estimated in the block coding distortion estimation step, 
and one of the two encoding modes that minimizes the sum is selected as 
the optimum encoding mode for the target block. 

In accordance with twenty-fifth through thirty-sixth aspects of the 
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present invention, there are provided machine-readable record mediums 
storing programs for instructing a microprocessor unit etc. to execute the 
motion video encoding methods of the thirteenth through twenty-fourth 
aspects of the present invention. 

5 

BRIEF DESCRIPTION OF THE DRAWINGS 
The objects and features of the present invention will become 
more apparent from the consideration of the following detailed 
description taken in conjunction with the accompanying drawings, in 
u 10 which: 

S Fig-l is a block diagram showing a conventional motion video 

jF encoding device; 

03 Figs.2A through 2C are schematic diagrams for explaining the 

Si relationship between the data loss probability and the point when 
pl5 transmission error occurred? 

r Fig.3 is a block diagram showing an example of the composition of 

Nj a motion video encoding device in accordance with a first embodiment of 
ft! the present invention; 

Figs.4A and 4B are schematic diagrams showing examples of 
20 processes for complementing the current frame with lost data; 

Fig. 5 is a schematic diagram for explaining the propagation of 
image degradation of the previous frame to the current frame; and 

Fig. 6 is a block diagram showing an example of the composition of 
a motion video encoding device in accordance with a second embodiment 
25 of the present invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Referring now to the drawings, a description will be given in 
detail of preferred embodiments in accordance with the present 
30 invention. 
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[Embodiment 1] 

Fig.3 is a block diagram showing an example of the composition of 
a motion video encoding device in accordance with a first embodiment of 
the present invention. The motion video encoding device of Fig.3 
includes a frame memory 10, a code volume control section 11, a 
degradation power calculation section 12, a data loss probability 
estimation section 13, a frame coding distortion estimation section 14, a 
degradation estimation calculation section 15, a memory 16, a mode 
selection section 17, a block encoding section 18, and a counter 19. 

The components shown in Fig.3 can be implemented by, for 
example, a microprocessor unit which is composed of a CPU (Central 
Processing Unit), ROM (Read Only Memory), RAM (Random Access 
Memory), etc., and appropriate software. Such software for realizing 
the operation of the motion video encoding device is stored in one or more 
record mediums and provided to the motion video encoding device so as 
to be loaded into the device. 

The code volume control section 11 refers to a statistic concerning 
pixel value distribution (obtained by analyzing an input image and a 
reference image which are stored in the frame memory 10) and an actual 
volume of code which has been necessary until the current block of the 
current frame (that is, code volume to a block just before the target 
block), and thereby sets a quantization step so that distortion caused by 
the encoding process will be minimum under the condition that the code 
volume for a frame should be a preset volume or less. The code volume 
control section 11 carries out the setting of the quantization step for each 
encoding mode that is selectable for the target block: the intra-frame 
encoding and the inter-frame prediction encoding. 
[Example of the Code Volume Control Method] 

The following explanation will be giving taking a method disclosed 
in Chapter 8 of "Video Codec Test Model, Near-Term, Version 10 
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(TMN10) Draftl" (published by ITU-T SG16), as an example of the code 
volume control method. Incidentally, even if other code volume control 
methods are employed, the quantization step setting and frame coding 
distortion estimation which will be explained later can be carried out and 
5 applied to the present invention. 

By describing the quantization step as "Q", pixel value variance in 
a block as " a 2 ", and the number of pixels in the block as "A", the volume 
(r) of code which is generated by encoding the block can be approximated 
as: 

HlO r=A-(K- aVQ 2 + C) • • • (l), 

a where K and C are proportionality constants. 

ffi In the above equation (l), the pixel value variance a 2 is given as 

L: the variance of a prediction error signal when the block has been encoded 
^ by means of the inter-frame prediction encoding, whereas the pixel value 
pl5 variance a 2 is given as 1/3 of the variance of the original video signal 
H; when the block has been encoded by means of the intra-frame encoding. 
O If we describe the sum total of a for blocks (as targets of 

EST = 

encoding) in a frame as "S", a code volume that should be assigned to the 
frame (target code volume) as "R", and the number of the blocks as 

20 targets of encoding as "N", a quantization step (Q op t) capable of 
minimizing the sum total of quantization distortion can be expressed as: 

Qopt = (A • K • a - S/L)" 2 ♦ • • (2), 
where L = R - A • N • C. 

The code volume control section 11 changes the quantization step 

25 so that the quantization step for each block will approach the optimum 
quantization step Q op t. For example, in "H.263" and "MPEG-4" as 
international standards for motion video encoding, the change of the 
quantization step from that of the previous block is limited to -4, -2, 0, +2 
or +4. The block encoding section 18 encodes the current block by use of 

30 the quantization step which has been set as above, while counting the 
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code volume actually needed until the point in time, and returns the 
counted code volume to the code volume control section 11. 

The code volume control section 11 distributes code volumes 
(executes bit allocation) to remaining blocks so that the sum of code 
volumes: a first code volume (that has actually been necessary till the 
current block of the current frame) and a second code volume (that will 
be necessary for the remaining blocks to be encoded), will reach the 
target code volume R. In this model, the sum total of the coding 
distortion for the blocks as targets of encoding (frame coding distortion 
Df) is estimated as: 

Df = A 2 • K • S 2 /12L • • • (3). 

The frame coding distortion estimation section 14 calculates the 
frame coding distortion Df by use of S (the sum total of a for the blocks 
as targets of encoding) which has been calculated by the code volume 
control section 11, and outputs the frame coding distortion Df to the 
mode selection section 17. Meanwhile, the code volume control section 
11 calculates a predicted value of the code volume for the next block 
(target block) according to the above equation (l) and sends the code 
volume predicted value to the data loss probability estimation section 13. 

The degradation power calculation section 12 assumes a case 
where transmission error occurred to coded data of a block, and 
calculates an error power (degradation power) "De" between a 
complemented image (an image complemented with lost data) and the 
original image. 

Fig.4A is a schematic diagram showing an example of a process 
for complementing the current frame with lost data. In the example of 
Fig.4A, transmission error is assumed to have occurred to coded data of 
blocks of the current frame. Lost data (pixel values) of the blocks of the 
current frame are complemented by directly copying pixel values from 
corresponding blocks of the reference image (reference frame) at the 
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same positions as the blocks of the current frame. The degree of image 
degradation in this case is expressed by the pixel value differences 
between the corresponding blocks of the reference image and the current 
frame. 

5 Fig.4B shows another example of a process for complementing the 

current frame with lost data. In the example of Fig.4B, lost data (pixel 
values) of the blocks of the current frame is complemented by copying 
pixel values from motion compensation areas of the reference image 
corresponding to the blocks. It is also possible to complement the lost 

MlO data (pixel values) of the blocks of the current frame with pixel values of 

□ surrounding pixels of the input image (current frame). 

S The degradation power calculation section 12 assumes such a 

complementing process by the decoder, generates a complemented image 
^ in the same way as the decoder by use of the reference image stored in 
O 15 the frame memory 10, and calculates an error power (degradation power 
H De) between the complemented image and the original image (current 
q frame). 

The data loss probability estimation section 13 estimates the 
probability of the occurrence of data loss in the target block due to errors 

20 (data loss probability "p") based on the code volume predicted value (LA) 
of the target block which is obtained by the code volume control section 
11 and a code volume (LB) from the latest synchronization code pattern 
to a block just before the target block (to a code sequence corresponding 
to a block that has been coded latest) which is counted by the counter 19. 

25 Describing LA + LB as "L0" and an error probability per bit (preset) as " 
X " and assuming random occurrence of errors, an equation for 
calculating the data loss probability p is given as: 

p = l-(l- X) L0 • • - (4). 
When X is very small, the above equation (4) can be approximated as p 

30 = X • L0. 
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The degradation estimation calculation section 15 refers to the 
degradation power De of the current frame which has been calculated by 
the degradation power calculation section 12, the data loss probability p 
of the target block of the current frame which has been calculated by the 
data loss probability estimation section 13, and a degradation 
estimations concerning reference blocks in the previous frame which 
have been stored in the memory 16, and thereby estimates the degree of 
degradation in case where error occurred to the target block (degradation 
estimation "Dd" of the target block) with regard to each encoding mode 
selectable for the target block (intra-frame encoding, inter-frame 
prediction encoding). 

The degradation estimation Dd of the target block varies 
depending on whether the encoding mode for the target block is the 
intra-frame encoding or the inter-frame prediction encoding. In the 
case of the intra-frame encoding, the degradation estimation Dd of the 
target block is expressed as the expected value of image degradation 
when data of the target block is lost due to errors, that is, as the product 
of the data loss probability p and the degradation power De (Dd = p • De). 
On the other hand, in the case of the inter-frame prediction encoding, the 
degradation estimation Dd of the target block is obtained by adding a 
propagation term "Dp" (indicating the propagation of image degradation 
of the reference frame due to the inter-frame prediction) to the product 
p • De (Dd = p • De + Dp). The propagation of image degradation will be 
explained referring to Fig. 5. 

Referring to Fig. 5, the reference character "BIO" denotes the 
target block of the current frame, and the reference character "B10 r " 
denotes an area (motion compensation area) of the reference frame that 
is referred to for motion compensation inter-frame prediction (hereafter, 
called "referred area"). Blocks BOO, B01, B02 and B03 of the reference 
frame overlapping with the referred area B10 f will be called "reference 



24 



blocks (BOO, B01, B02 and B03)", and parts of the reference blocks BOO, 
B01, B02 and B03 included in the referred area BIO' will be described as 
areas "bOO", "bOl", "b02" and "b03", respectively. A motion vector for the 
target block BIO will be described as a vector "V". 

By the inter-frame prediction encoding, the target block BIO 
inherits the entire image degradation of the referred area BIO'. The 
image degradation of the referred area BIO' is equal to image 
degradation of the areas bOO, bOl, b02 and b03 added together. The 
image degradation of each area (bOO, bOl, b02, b03) is approximated by 
multiplying the image degradation of a corresponding reference block 
(BOO, BOl, B02, B03) by an area ratio. 

The area ratio can be obtained by use of the motion vector V. If 
we describe the length (width) of each block as "m", the length of the 
projection of the motion vector V on the horizontal axis (x-axis) as "Vx", 
and the length of the projection of the motion vector V on the vertical 
axis (y-axis) as "Vy", the area ratio of the area bOO to the reference block 
BOO can be expressed as (m - Vx) • (m - Vy)/(m • m). Area ratios for the 
other areas bOl, b02 and b03 can be obtained similarly. 

Therefore, the degradation propagation term Dp of the target 
block BIO inherited from the reference frame can be obtained by 
referring to degradation estimations of the reference blocks BOO, BOl, 
B02 and B03 which have been stored in the memory 16, multiplying each 
degradation estimation by a weight according to the motion vector V, and 
adding the weighted degradation estimations together. 

The mode selection section 17 compares the sum (Dd (degradation 
estimation) + Df (frame coding distortion)) when the encoding mode for 
the target block is the intra-frame encoding with the sum (Dd + Df) when 
the encoding mode for the target block is the inter-frame prediction 
encoding, and selects one of the encoding modes that minimizes the sum 
(Dd + Df). The block encoding section 18 encodes the target block 
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according to the encoding mode selected by the mode selection section 17 
and the quantization step calculated by the code volume control section 
11, thereby coded data for the target block is generated. 

Incidentally, while the degradation estimation Dd in the 
5 inter-frame prediction encoding was obtained in the above explanation 
by simply adding the degradation propagation term (Dp) inherited from 
the previous frame to the degradation expected value (p • De) in the 
current frame, it is also possible to assign weights to the degradation 
propagation term (Dp) from the previous frame and/or the degradation 
ulO expected value (p • De) in the current frame and change the weights 
O depending on image quality adjustment in the designing stage. 
m For example, by weighting the degradation propagation term (Dp) 

2 from the previous frame with a factor larger than 1, the intra-frame 
^ encoding is preferentially carried out to blocks having large degradation 
□ 15 propagation from the previous frame, thereby stagnation of degraded 
U blocks across frames can be avoided. In addition, it is also possible to 
p let the mode selection section 17 execute the mode selection so as to 
minimize the degradation estimation Dd of the target block only, without 
the consideration to the frame coding distortion Df. In this case, coding 
20 efficiency deteriorates, however, encoding having optimum resistance to 
errors can be realized. 
[Embodiment 2] 

Fig.6 is a block diagram showing an example of the composition of 
a motion video encoding device in accordance with a second embodiment 
25 of the present invention. The motion video encoding device of Fig.6 
further includes a block coding distortion estimation section 21, in 
comparison with the first embodiment of Fig. 3. 

The mode selection section 20 shown in Fig.6 operates a little 
differently from the mode selection section 17 of Fig. 3. The mode 
30 selection section 20 refers to block coding distortion Db which is 
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calculated by the block coding distortion estimation section 21 in 
addition to the degradation estimation Dd of the target block calculated 
by the degradation estimation calculation section 23 and the frame 
coding distortion Df calculated by the frame coding distortion estimation 
section 14, and thereby selects an encoding mode from three encoding 
modes- the intra-frame encoding (INTRA); the inter-frame prediction 
encoding (INTER); and "skip" (SKIP). 

When the encoding mode selected by the mode selection section 20 
is SKIP, the block encoding section 18 does not encode motion 
information nor coefficient information with regard to the target block 
but outputs a skip code (code pattern indicating that the encoding mode 
is SKIP) only. In this case, the decoding of the target block of the 
current frame is carried out by the decoder by copying decoded data of a 
corresponding block of the previous frame. 

The block coding distortion estimation section 21 estimates the 
coding distortion Db of the target block (block coding distortion) for each 
encoding mode (intra-frame encoding, inter-frame prediction encoding, 
skip). 

When the encoding mode is the intra-frame encoding or the 
inter-frame prediction encoding, the coding distortion Db is equal to the 
error power caused by quantization. In this case, the block coding 
distortion estimation section 21 estimates the coding distortion Db of the 
target block by referring to the quantization step in each encoding mode 
(intra-frame encoding, inter-frame prediction encoding). For example, 
when the quantization is carried out by linear quantization, the coding 
distortion Db of the target block is estimated as Db = Q 2 /12 • A (Q: 
quantization step, A: the number of pixels in the block). 

On the other hand, when the encoding mode is SKIP, decoded 
image of the target block of the current frame will be the same as 
decoded image of a corresponding block (target block) in the previous 
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frame, therefore, the coding distortion Db is expressed as error power 
between the previous frame and the current frame. In this case, the 
block coding distortion estimation section 21 obtains the error power 
between the previous frame and the current frame and regards the error 
5 power as the coding distortion Db of the target block. 

For the encoding mode SKIP, the data loss probability estimation 
section 22 and the degradation estimation calculation section 23 shown 
in Fig. 6 also operate differently from the data loss probability estimation 
section 13 and the degradation estimation calculation section 15 of the 
10 first embodiment. When the encoding mode is SKIP, the data loss 
probability estimation section 22 estimates the data loss probability p of 
the target block, based on the code volume necessary for the skip code 
(indicating that the encoding mode is SKIP) and the code volume counted 
I by the counter 19 (code volume from the latest synchronization code 
, 15 pattern to a block just before the target block). The degradation 
estimation calculation section 23 calculates the degradation estimation 
Dd for the encoding mode SKIP based on the data loss probability p 
obtained by the data loss probability estimation section 22, in the same 
way as the case where the encoding mode is the inter-frame prediction 
20 encoding. 

The mode selection section 20 first selects a candidate encoding 
mode from the intra-frame encoding and the inter-frame prediction 
encoding in the same way as the mode selection section 17 of the first 
embodiment, thereafter compares the selected candidate encoding mode 

25 with the encoding mode SKIP by making a comparison between the two 
modes with regard to (Dd + Db) (that is, the degradation estimation Dd 
calculated by the degradation estimation calculation section 23 and the 
block coding distortion Db estimated by the block coding distortion 
estimation section 21 added together), and selects one of the two 

30 encoding modes that minimizes the sum Dd + Db as the optimum 
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encoding mode for the target block. 

As set forth hereinabove, in the motion video encoding device and 
the motion video encoding method in accordance with the present 
invention, the encoding mode selection is carried out so that the sum of 
the error-caused degradation and the coding distortion will be minimum. 
By such encoding mode selection for each block, the forced refresh can be 
carried out properly and effectively and thereby image degradation can 
be reduced without deteriorating the coding efficiency. 

Further, in the present invention, the degree of image degradation 
caused by transmission error is estimated for each block, and the 
intra-frame encoding is carried out with high priority to blocks that are 
expected to cause image degradation. The frequency of the refresh has 
been determined and adjusted empirically in conventional methods, 
whereas proper refresh can be realized constantly by the present 
invention. 

Further, in the present invention, the mode selection is carried 
out by use of the code volume predicted value (LA) of the target block 
and the estimate of the coding distortion. Therefore, proper mode 
selection for each block can be carried out concurrently with the encoding 
in units of blocks, and the amount of necessary operations can be 
reduced to minimum. 

In addition, the present invention employs the encoding mode 
SKIP in addition to the intra-frame encoding and the inter-frame 
prediction encoding, as a candidate for the encoding mode selection in 
consideration of resistance to errors. The code volume occurring when 
the encoding mode SKIP is selected is as small as 1 bit and the 
probability of data loss due to errors becomes lower in comparison with 
other encoding modes, therefore, the resistance to errors can be improved 
further. 

While the present invention has been described with reference to 
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the particular illustrative embodiments, it is not to be restricted by those 
embodiments but only by the appended claims. It is to be appreciated 
that those skilled in the art can change or modify the embodiments 
without departing from the scope and spirit of the present invention. 



